Skip to content

Move common docs upstream #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

SJaffa
Copy link
Collaborator

@SJaffa SJaffa commented May 8, 2025

We have accidentally been improving the MatFlow docs and ignoring the HPCflow ones. Any changes that are also relevant to HPCflow should be in this repo so they are populated to both docs.

stop: 32
parallel_mode: null

- name: dream_3D_env
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if Dream3D is a material science specific tool or general enough to be relevant to HPCflow users too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll only want to include the python_env example here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we need to update: command: python <<script_name>> <<args>> to command: python "<<script_path>>" <<args>>

@SJaffa SJaffa linked an issue May 8, 2025 that may be closed by this pull request
@SJaffa SJaffa marked this pull request as ready for review May 8, 2025 15:13
@SJaffa SJaffa self-assigned this May 8, 2025
@SJaffa SJaffa requested a review from aplowman May 8, 2025 15:15
@SJaffa SJaffa mentioned this pull request May 8, 2025
Copy link
Contributor

@aplowman aplowman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few things I must have missed when they went into MatFlow!

stop: 32
parallel_mode: null

- name: dream_3D_env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll only want to include the python_env example here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't include this file in the shared docs because hpcflow has an additional page that must be linked from here (SDK).

stop: 32
parallel_mode: null

- name: dream_3D_env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we need to update: command: python <<script_name>> <<args>> to command: python "<<script_path>>" <<args>>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this file needs to be within a.. jinja:: first_ctx block.

Comment on lines +17 to +21
scheduler_args:
shebang_args: --login
options:
-l: short

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be updated as per hpcflow/hpcflow-new#805. So:

Suggested change
scheduler_args:
shebang_args: --login
options:
-l: short
shell_args:
executable_args: ["--login"]
scheduler_args:
directives:
-l: short

options:
-l: mem512

Anything specified under `options` is passed directly to the scheduler as a jobscript command (i.e. isn't processed by {{ app_name }} at all).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Anything specified under `options` is passed directly to the scheduler as a jobscript command (i.e. isn't processed by {{ app_name }} at all).
Anything specified under `directives` is passed directly to the scheduler as a jobscript command (i.e. isn't processed by {{ app_name }} at all).

Comment on lines +92 to +95
main:
num_cores: 16
scheduler_args:
options: {} # "Clear" any previous options which have been set.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
main:
num_cores: 16
scheduler_args:
options: {} # "Clear" any previous options which have been set.
main:
num_cores: 16
scheduler_args:
directives: {} # "Clear" any previous directives which have been set.

- my_input_2
script: <<script:/full/path/to/generate_input_file.py>>

An example is given in [advanced_workflow.yaml](advanced_workflow.yaml), along with the alternative code which would be needed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a markdown URL.

Modify the paths to the python scripts under the ``action`` keys to give the full path
to your files.

You can then run the workflow using ``{{ app_module }} go workflow.yaml``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can then run the workflow using ``{{ app_module }} go workflow.yaml``.
You can then run the workflow using ``{{ app_package_name }} go workflow.yaml``.


This is usually caused by updating the {{ app_name }} version.
Leftover submissions info causes the newer {{ app_name }} version to get confused.
The fix? ``{{ app_module }} manage clear-known-subs``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The fix? ``{{ app_module }} manage clear-known-subs``.
The fix? ``{{ app_package_name }} manage clear-known-subs``.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move relevant docs out of downstream repos
2 participants